Tuning Variable Selection Procedures by Adding Noise

Many variable selection methods for linear regression depend critically on tuning parameters that control the performance of the method, e.g., “entry” and“stay” significance levels in forward and backward selection. However, most methods do not adapt the tuning parameters to particular data sets. We propose a general strategy for adapting variable selection tuning parameters that effectively estimates the tuning parameters so that the selection method avoids overfitting and underfitting. The strategy is based on the principle that underfitting and overfitting can be directly observed in estimates of the error variance after 1) adding controlled amounts of additional independent noise to the response variable and 2) then running a variable selection method. It is related to the simulation technique SIMEX found in the measurement error literature. We focus on forward selection because of its simplicity and ability to handle large numbers of explanatory variables. Monte Carlo studies show that the new method compares favorably with established methods. Xiaohui Luo is Biometrician, Clinical Biostatistics, Merck Research Laboratories, Rahway, NJ 07065-0900 (Email: edmund [email protected]). Leonard A. Stefanski and Dennis D. Boos are Professors, Department of Statistics, North Carolina State University, Raleigh, NC 27695-8203 (E-mail: [email protected], [email protected]). 1